Cross-Media Retrieval using Probabilistic Model of Automatic Image Annotation

In recent years, automatic image annotation (AIA) has been applied to cross-media retrieval usually due to its advantage of mining correlations of images and annotation texts efficiently. However, some AIA methods just annotate images as a unit and the accuracy of annotation may not be acceptable. In this paper, we propose a kind of probabilistic model which may assign keywords to an un-annotated image automatically based on a training dataset of images. Images in the training dataset are segmented into regions and a kind of vocabulary called blob is used to represent these image regions. Blobs are generated by using K-Means algorithm to cluster these image regions. Through this model, we can predict the probability of assigning a keyword into a blob. After the accomplishment of annotation, a keyword corresponds to one image region. Furthermore, the feature vectors of text documents are generated by TF.IDF method and images’ automatic annotation information is used to retrieve relevant text documents. Experiments on the IAPR TC-12 dataset and 500 Wikipedia webpages about landscape show the usefulness of applying probabilistic model of AIA to the cross-media retrieval.